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Abstract 

Background: Point-of-care testing (POCT) is increasingly being used in general practice to assist general 
practitioners (GPs) in their management of patients with diseases. However, low adherence to quality guidelines in 
terms of split test procedures has been observed among GPs in parts of the Capital Region in Denmark. Computer 
reminders embedded in GPs electronic medical records (ComRem) may facilitate improved quality control 
behaviour, but more research is needed to identify what types of reminders work and when. The overall aim of 
this study is to evaluate the efficacy of ComRem to improve GPs adherence to quality guidelines. This article 
describes the rationale and methods of the study that constitute this research project. 

Methods/design: The study is conducted as two randomised controlled trials (RCTs) among general practices in 
two districts of the Capital Region in Denmark. These districts contain a total of 739 GPs in 567 practices with a 
total of 1.1 million patients allocated to practice lists. In the first RCT (RCT A), ComRem is compared to postal 
reminder letters. In the second RCT (RCT B), ComRem is compared to usual activities (no reminders) with a 
crossover approach. In both of these studies, outcomes are measured by the number of split tests received by the 
laboratory. 

Conclusions: This study will contribute to knowledge on the efficacy of ComRem in primary care. Because the 
study does not explore GPs' perceptions and experiences with regard to ComRem, we will subsequently conduct a 
qualitative survey focusing on these aspects. 

Trial registrations: Study A: ClinicalTrials.gov identifier: NCT01 152151 
Study B: ClinicalTrials.gov identifier: NCT01 1 521 77 



Background 

Point-of-care testing (POCT) is increasingly being used 
in general practice to assist general practitioners (GPs) in 
their daily work with patients. For adequate deployment 
of POCT, an external quality assessment (EQA) is 
recommended on a monthly basis [1]. In the Copenhagen 
area, EQA is enforced by a split test procedure as well as 
annual outreach consultant visits. In a split test, the 
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actual POCT result is compared to a result from a blood 
sample from the same individual analyzed at the central 
laboratory. The quotient of these two results should ide- 
ally be 1.00, but a value inside the interval ranging from 
0.85 to 1.15 is acceptable [1]. This quotient is returned to 
the practice for self- evaluation. However, the adherence 
to the monthly split test procedure has not been satisfac- 
tory among GPs in two districts of the Capital Region 
(Table 1). Therefore, the Copenhagen General Practi- 
tioners' Laboratory (hereafter simply referred to as 'the 
laboratory') planned to improve adherence. 
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Table 1 GP and practice characteristics and distribution of the Point of Care tests at baseline (January to April 2010) 
stratified on the two RCTs 



Study A 



Study B 



Computer reminder/postal reminder 
(Practices = 213) 
(GPs = 318) 
Mean (SD) or n (%) 



Computer reminder/usual care 
(Practices = 286) 
(GPs = 341) 
Mean (SD) or n (%) 



GP and practice characteristics 



Gender 


Male 


1 77 (56) 


171 (50) 




Female 


1 A 1 (AA\ 


1 / U 


Mean age (SD) 




53.1 (7.8) 


56.3 (8.2) 


Mean years as GP 




1 1 .3 (8.8) 


14.8 (9.6) 


Total number of patients on practice lists 




507,568 


533,864 


Mean number of patients per GP (SD) 




1,596 (419) 


1,566 (398) 


Practice organization Single 


Handed 


139 (65) 


239 (84) 




Group 


74 (45) 


47 (16) 


Point of Care Tests at baseline 








Mean INR tests 




86 (58) 




Mean Hemoglobin tests 




63 (97) 


40 (55) 


Mean Glucose tests 




78 (92) 


47 (52) 


Primary outcome 




INR* 


HGB/GLU** 


Average number of split tests 




0.35 (0.55) 


1.07 (1.36) 


Secondary outcomes 








Practice with three or four split tests 




0 (0) 


0 (0) 


Practices with at least one split test 




67 (31) 


146 (51) 



*four recommended in baseline period 

**four are recommended for each of the two tests in baseline period 



Dissemination of guidelines alone rarely brings about 
improvements in clinical practice [2], and even an mul- 
tifaceted implementation of guidelines may not change 
clinical practice [3,4]. Multiple strategies for implement- 
ing guidelines appear to be more effective than single 
ones [5,6]. However, when evaluated rigorously, these 
strategies often produce only minor benefits and rarely 
on patient outcomes [7,8]. In general, well-designed 
empirical research looking into various implementation 
strategies is still needed in this area [9]. In order to 
improve adherence, the laboratory wants to employ 
computer reminders embedded in the GPs' electronic 
medical records (ComRem) additionally to the quality 
control enforcement activities. At the same time, the 
laboratory wanted to conduct a rigorous evaluation of 
this innovative approach. 

Within the last decade, several systematic reviews have 
evaluated computer reminders [10-13]. However, these 
reviews have pooled several different types of computer 
reminders, e.g., computer-generated paper reminders, 
and e-mail alerts along with reminders generated at 
point of care. According to a recent Cochrane review, 
computer reminders have been shown to produce a 
small improvement on target behaviour [13]. The review 



concludes that more research is needed to identify what 
types of reminders work and when. 

The overall aim of the randomised controlled trials 
(RCTs) described in this paper is to evaluate the efficacy 
of ComRem on adherence to clinical quality guidelines 
regarding POCT. This aim is translated into the follow- 
ing research questions: 

1. What is the efficacy of ComRem compared to com- 
puter-generated postal reminder letters on general prac- 
tices' adherence to clinical quality guidelines regarding 
POCT (RCT A)? 

2. What is the efficacy and legacy of ComRem com- 
pared to usual activities (i.e., no special reminders) on 
GP adherence to clinical quality guidelines regarding 
POCT (RCT B)? 

The reason why the study is divided into two RCTs is 
the laboratory's priority to implement a quality improve- 
ment intervention for all practices conducting interna- 
tional normalized ratio (INR) analyses. Hence, these 
practices could not be allocated to usual practice alone. 

Methods 

The study is conducted as two RCTs among the prac- 
tices in two districts in the Capital Region in Denmark. 
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These districts contain a total of 739 GPs in 567 
practices. 

Usual EQA practice 

The guidelines recommend a split test procedure each 
month for each POCT instrument. The standard imple- 
mentation of EQA consists of an annual facilitator visit 
in each practice. Also, information about the split tests 
was posted in three laboratory newsletters. Furthermore, 
an instruction on how the practices should execute split 
tests was posted as a link on the laboratory's web site. 

Computer reminders embedded in the GPs' electronic 
medical records (ComRem) 

In the Danish health care sector, a common standard 
(MedCom) for electronic communication has been 
defined and used for a decade. MedCom is a co-operative 
venture between authorities, organizations, and private 
firms linked to the Danish healthcare sector, and uses a 
Danish adapted format of Electronic Data Interchange 
For Administration, Commerce and Transport (UN/EDI- 
FACT) originally developed by the United Nations. Med- 
Com offers secure electronic communication between 
hospitals, pharmacies, laboratories, and primary care, e.g., 
GPs and municipalities. Each practice has been allocated 
to a unique location number according to the MedCom 
standard. All electronic communication is delivered into 
a specific practice inbox and each communication has to 
be approved by the GP. 

Participants 

Included in the study are all general practices that had 
performed at least four relevant POCT-INR, hemoglo- 
bin, or glucose-in the baseline study period (1 January 
to 31 April 2010) and had access to POCT within their 
own practice. These practices were identified in the GP 
database of the Capital area and in the laboratory 
database. 

Data collection 

Data on performed split test procedures is retrieved 
from the laboratory database. These data do not contain 
any patient-related data because all split tests are con- 
ducted by an artificial identification code. Process indi- 
cators (sent reminder letters) are also obtained from the 
laboratory. The Capital Region databases provide infor- 
mation on the participating practices and corresponding 
GPs. 

Randomization 

Within each study, practices are randomized into two 
similarly sized groups by means of computer generated 
random numbers using SAS version 9.2. To ensure that 
practice types are distributed equally over the intervention 



and control arms, randomization within each RCT is done 
separately for group practices and solo practices respec- 
tively. This randomization was conducted by the data 
manager of the Research Unit of General Practice without 
knowledge of the individual practice identification. 

Data from the capital region and Copenhagen general 
practice laboratory 

The study period comprises a four-month baseline period 
before the start of the intervention and up to three four- 
month follow-up periods after the start of the intervention. 
During the study periods, the number of specific POCT 
tests done in each practice is monitored using the relevant 
bills sent to the Capital Region by the GPs. Concurrent, 
the occurrence of split tests on the specific POCT analyses 
and-for practices participating in an intervention arm-the 
number of ComRem or postal reminders sent in each of 
the follow-up periods are monitored in the laboratory 
database. Combined, this monitoring gives series of up to 
four measures (defined by several outcomes, see below) of 
adherence to the split test procedures-for the baseline 
period and for each of the follow-up periods-for each 
practice. Background information on practices and GP 
was retrieved by the Capital Region at the start of the 
study period. This information includes gender, age, year 
of graduation from university, working address, type of 
practice, and list size. 

Outcomes 

Outcomes are measures of split test procedure adher- 
ence that are calculated from the monitor data. 

Primary outcome 

Total number of split test procedures for the correspond- 
ing POCT analysis performed by the practice in a four- 
month period. In a given month, a split test procedure 
should only be performed if the practice conducted the 
POCT analysis. The maximum number of possible split 
test procedures for a single analysis in a four-month per- 
iod is four. 

Secondary outcomes 

1. Whether the practice has a high quality of tests 
defined as 75% of the performed split test procedures 
within the reference interval (according to the laboratory 
quality guidelines). 

2. Whether split test procedures are performed at all 
by the practice. 

Statistics 

Cross-sectional differences in the outcomes between allo- 
cation groups in the baseline period and each of the fol- 
low-up periods are tested by means of t-tests (primary 
outcome) and chi-square tests (secondary outcomes). In 
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order to investigate the development of split test proce- 
dure adherence over the up to four periods relative to the 
(changing) intervention, Poisson (primary outcome) and 
logistic (secondary outcomes) regression is used with GEE 
methods to account for the repeated measurements. To 
identify predictors for adhering to split test procedure 
guidelines, adjusted effects for the GP and practice charac- 
teristics are estimated in multivariate Poisson (primary 
outcome) and logistic (secondary outcomes) regression 
analysis on the outcomes at selected periods. Beyond the 
direct comparison of the randomization groups in the first 
follow-up period, comparisons in further follow-up peri- 
ods are done to investigate various forms of potential 
legacy effect, i.e., the effect of an intervention in periods 
where the intervention is discontinued. All statistical ana- 
lyses are performed using SAS, version 9.2 (SAS Institute 
Inc, Cary, NC). 

RCT A: Comparing computer reminders with postal 
reminders 

Included in RCT A are the practices conducting more 
than four POCT INR during the four-month baseline 
period (213 practices). Practices are randomly allocated 
to computer-generated postal reminder letters or to 
ComRem; both in addition to usual laboratory quality 
guideline activities. The comparison of outcomes after a 
first follow-up period: the intervention period estimates 
the relative efficacy of ComRem over postal reminder 
letters; a comparison after a second follow-up period in 
which both interventions are discontinued estimates the 
relative legacy effect of the interventions. 

Intervention 

Postal reminder letters 

In this group, computer-generated postal reminder let- 
ters are sent to practices not adhering to the guideline 
recommendations of split testing within the previous 
calendar month. Thus, practices may receive up to four 
postal reminder letters during each four-month follow- 
up round. The contents of the reminder are presented 
in Figure 1. 

ComRem 

In this group, ComRem are sent to practices not adher- 
ing to the guideline recommendations of split testing 
within the previous calendar month. Thus, practices 
may receive up to four ComRem. These reminders have 
exactly the same content as the postal reminders. 

Outcomes 

The outcomes specified previously address the conduct 
of POCT for INR. The study period comprises three 
four-month periods: The baseline, the intervention, and 
follow up period. 



Power calculation 

We used an estimate of a mean number of 1.5 INR split 
tests based on laboratory data from 2007 in order to 
ascertain the power of the study. Given a standard devia- 
tion of 1, a power of 90%, and an effect of 0.5, we esti- 
mated that 172 practices are to be included in this study. 

RCT B: Comparing computer reminders with usual 
activities and the potential legacy of electronic reminders 

Included in the study are the 286 general practices con- 
ducting at least four POCT (either hemoglobin or glucose) 
during the four-month baseline period and that are not 
already included in RCT A. Practices are allocated to 
ComRem together with usual laboratory EQA practice or 
usual laboratory EQA practice only. The comparison of 
outcomes after a first follow-up period, the intervention 
period, estimates the relative efficacy of ComRem versus 
usual care only; a comparison after a second follow-up 
period in which the interventions are switched between 
groups estimates the relative efficacy of a (short-term) 
legacy effect of ComRem over the direct effect of Com- 
Rem; a comparison after a third follow-up period in which 
both interventions are discontinued estimates the relative 
efficacy of a long-term legacy effect of ComRem over the 
short-term legacy effect of ComRem. 

Intervention 

ComRem 

Computer reminder letters are sent to practices not 
adhering to the guideline recommendations of split test- 
ing for either hemoglobin or glucose within the previous 
calendar month. Thus, all practices may receive up to 
four electronic reminder letters in each period. The con- 
tent of the reminder is identical to the one in RCT A, 
except that this reminder addresses hemoglobin/glucose 
and not INR. 

Outcome 

The outcomes specified previously address the conduct 
of POCT for hemoglobin and glucose. The study period 
comprises four four-month periods: The baseline, the 
intervention, cross-over, and follow up period. 

Power calculation 

We used an estimate of a mean number of one hemo- 
globin and/or glucose split test based on laboratory data 
from 2007 in order to ascertain the power of the study. 
Given a standard deviation of 1.25, a power of 90% and 
an effect of 0.5, we estimated that 266 practices are to 
be included in this study. 

Ethics 

This study uses blood samples that are not retraceable 
to specific patients in order to conduct the split tests. 
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Copenhagen, DD.MM.YYYY 

Dear Colleague, 

Within the last calendar month we have not detected an external quality 
assessment analysis for INR from your practice point of care equipment. 



Execution of external quality assessment analysis is important in order to 
improve the quality of the point of care testing and is a prerequisite for 
receiving fees for this service. Apply number 010158-6006 for ordering 
external quality assessment analysis in WebQuality system. 

For questions, please contact our telephone hotline: 3374 41 31 . 

If you do not perform INR, please disregard this message. 



1 



Yours sincerely, 
Niels Mosbaek, 
Administration officer, CGPL 



Figure 1 Contents of reminder (RCT A) 



Patient identities are kept hidden through an artificial 
identification code used by all practices. The project has 
been evaluated by the Scientific Ethical Committee for 
Copenhagen and Frederiksberg Municipalities (j. nr. H- 
1-2010_FSP/10) and the Danish Data Protection Agency 
(j. nr. 2010-41-4680). The Danish College of General 
Practitioners Study Committee evaluated the project 
(MPU 12-2010). The two RCT are registered in Clinical- 
Trials.gov Identifier (NCT01152151, NCT01152177). 

Results 

A total of 567 practices with a total of 739 GPs were eli- 
gible in the study area with a total population of 1.1 
million. In total, 213 practices were included in RCT A 
and 286 were included in RCT B (Figure 2). The charac- 
teristics of GPs and practices are presented in Table 1. 
Split tests were in general infrequently used by the prac- 
tices (Table 1). 

Discussion 

This paper describes the rationale and design of two RCTs 
testing the efficacy of ComRem on GPs' adherence to 
quality guidelines on POCT. Thus, our study is in accor- 
dance with a recent Cochrane Review that calls for more 
research into ComRem in order to identify the features 
associated with improvements in provider behavior [13]. 



This is the first study in general practice to address 
computer reminders. All practices in the two districts 
are eligible, and only practices with few POCT or with 
no POCT equipment are excluded. Thus, the study 
population represents behavior among those who routi- 
nely use POCT in their clinical decisions. The two 
RCTs studies are designed to comply with the recom- 
mendations of the CONSORT statement [14]. The data 
manager conducted the randomization without knowing 
the practice code. The participating practices are una- 
ware of their randomization group allocation up to the 
point when the first reminder (electronic or postal) does 
or does not come in. Because the study is incorporated 
in the activities of the laboratory, many practices may 
not realize that they are participating in the RCTs. 
Thus, the results from this study represent the present 
daily standard. Blinded assessment of study outcomes is 
not relevant, because outcome data is retrieved from the 
laboratory's electronic data system that documents 
delivery of the split tests as well as routine data from 
the Capital Region [13]. Because not much is known on 
the efficacy of ComRem on POCT quality, it was diffi- 
cult to do a power analysis in the design phase of the 
studies. Hence, we based the power analysis on a labora- 
tory report from 2007 and experiences with postal 
reminders [15]. 
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Jan-Apr 2010 



Sep-Dec 2010 



Jan-Apr 2011 



May-Aug 2011 




Figure 2 Trial flow for the project 



In the two RCTs we evaluate different approaches. In 
RCT A we compare computer reminders with postal 
reminders, and in RCT B we compare computer remin- 
ders with usual activities. The two RCTs are conducted 
simultaneously, without any redundancy between prac- 
tices. The Capital Region and the laboratory had before- 
hand agreed to focus on INR. Due to the seriousness of 
producing a wrong INR analysis result, it was demanded 
that both arms should have an intervention involving 
prompting, because any prompting most likely would 
increase the adherence rate. Hence, no comparison with 
usual laboratory EQA practice only was done in RCT A. 
RCT B offers the possibility to survey a legacy effect, 
which is how long one should expect an effect of compu- 
ter reminders to last. This aspect has not previously been 
studied. In addition to the main aims, we will have the 
possibility to measure cost effectiveness as well as indir- 
ectly derived efficacy on other POCT analyses e.g., the 
effect on c-reactive protein split tests. 

The EQA procedures for POCT here described reflect 
the standard in Denmark. Other countries' EQA proce- 
dures for POCT may not use prompting, and our 
study's results do not generalize to them. However, 
POCT may be viewed as a case of more general prompt- 
ing, and our study's results may be generalized to other 
systems where prompting is used. 



In order to advance our interpretation of the results 
and our understanding of how the electronic reminders 
relate to beliefs, technologies, and routines in general 
practice, a qualitative study using semi- structured inter- 
views [16] will be conducted after the two RCTs are com- 
pleted. Linking qualitative methods to RCT studies is 
increasingly recommended in the field of health services 
research [17] where qualitative methods have been 
applied before, during or after randomized controlled 
trials [18]. The key questions to be explored in the 
planned qualitative study are: How is quality control con- 
cerning POCT organized in the clinics and how do the 
GPs perceive the request for quality assurance for 
POCT? How do GPs regard the use of ComRem to pro- 
mote adherence to quality assurance guidelines? What 
were the considerations and responses in the clinics {e.g., 
changes in procedures, roles, and responsibilities) follow- 
ing the reception of a ComRem? And how can variations 
in responses to ComRem be explained? Respondents for 
the interviews will be strategically selected [19] based on 
the results concerning the primary outcomes of the inter- 
vention (change versus no change in the number of split 
tests performed after ComRem). 

On the basis of the quantitative and qualitative analy- 
sis, it will be considered if and how adjustments can be 
made to the content and employment of ComRem in 
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order to support the ambitions of EQA in balance with 
other legitimate concerns of general practice. 

In summary, research into computer reminders has 
been called for by a recent Cochrane review, and the 
present study focuses on primary care. This study will 
generate new knowledge on the efficacy and limitations 
of computer reminders in primary care. 

End Notes 

Practices not conducting neither four INR tests nor 
four Hemoglobin/Glucose test in jan-april 2010 or did 
not have POCT; Practices conducting four or more 
INR test in jan-april 2010; 3 Not conducting four INR 
test, but more than four Hemoglobin/Glucose test in 
jan-april 2010. 
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